1. Technical Field
The invention relates to systems for detecting and tracking targets, such as touches or movements by a user or user tool, on or near a surface. In particular, the invention relates to systems capable of detecting and tracking multiple targets simultaneously on a user interface surface using triangulation.
2. Related Art
In many situations, where computers are being accessed, used, or seen by more than one person, it may be useful to allow multiple users to input data to the system at the same time. The situations may be in a business setting or in a consumer-oriented setting.
Users may input data into a computer system in a variety of ways. Conventional examples include a keyboard, a mouse, a joystick, etc. Another conventional type of user input is a touchable display. Specifically, a user may touch a part of a display screen, and the location of the user's touch on the screen may then be sent to an application program for processing of the user's input.
Touchable displays allow one or more users to interact directly with the screen, either in conjunction with or instead of a mouse, pointer, or the like. There are many applications for touchable displays including, for example, Automated Teller Machines (ATMs), airport kiosks, manufacturing input, etc. These displays include a sensing system that passes state information as to the user's touch or interaction to application software that interprets the state information in the context of the application.
Touchable displays may use various types of sensors to detect a touch. One type of touchable display may sense changes in capacitance due to the presence of a touch on the screen. The screen may be segmented into a plurality of contact areas so that when a user touches a specific contact area, circuitry associated with the contact area may sense a change in capacitance to determine the touch. The touchable display may multiplex, or step through, sensing whether each of the plurality of contact areas has been touched. In this manner, the capacitive touchable display may sense both single and multiple touches.
Another type of touchable display may use a camera, or other type of light sensor (e.g., visual and/or infrared light sensors), to detect a touch on the screen. Camera-based touchable displays are typically used with larger-sized displays. An exemplary camera-based system is disclosed in U.S. Patent Application No. 2005/0077452A1 (U.S. application Ser. No. 10/312,983), published on Apr. 14, 2005, which is incorporated by reference. In these systems, two or more cameras may look along the length of and parallel to the surface of the screen. The cameras are thus positioned to sense a target (such as a touch from a finger, an elbow, a pen, or any other object) proximate to or touching the surface. In order for the cameras to better sense the target, a contrasting material, such as black tape or other dark material, may be placed opposite the cameras. Thus, when a user touches the screen, the cameras sense the single target as a single bright spot within each camera's field of view.
Each camera produces a two-dimensional output (i.e., a picture with no depth information) so that the target may be at any position along a ray from the focal point of the camera, through the target, and beyond. Because multiple cameras sense the target, it is possible to triangulate the position of the single target on the surface by: (1) determining the ray for each camera; and (2) determining the intersection of the two or more rays. This triangulation, thus, provides the position of the single target on the surface.
Unlike capacitive touchable systems, the camera-based systems that use triangulation are limited to detecting a single target on the screen. In the event that a second part of the screen is touched simultaneously or concurrently, the camera based system would have undefined or undesirable effects. This is due to the inability of the cameras to sense depth information and the limits of using triangulation to discern the position of touches.
Thus, there is a need for a camera-based system to detect and track multiple simultaneous targets using triangulation.